15 research outputs found
Consistent Semantic Annotation of Outdoor Datasets via 2D/3D Label Transfer
The advance of scene understanding methods based on machine learning relies on the availability of large ground truth datasets, which are essential for their training and evaluation. Construction of such datasets with imagery from real sensor data however typically requires much manual annotation of semantic regions in the data, delivered by substantial human labour. To speed up this process, we propose a framework for semantic annotation of scenes captured by moving camera(s), e.g., mounted on a vehicle or robot. It makes use of an available 3D model of the traversed scene to project segmented 3D objects into each camera frame to obtain an initial annotation of the associated 2D image, which is followed by manual refinement by the user. The refined annotation can be transferred to the next consecutive frame using optical flow estimation. We have evaluated the efficiency of the proposed framework during the production of a labelled outdoor dataset. The analysis of annotation times shows that up to 43% less effort is required on average, and the consistency of the labelling is also improved
DUGMA: Dynamic Uncertainty-Based Gaussian Mixture Alignment
Registering accurately point clouds from a cheap low-resolution sensor is a
challenging task. Existing rigid registration methods failed to use the
physical 3D uncertainty distribution of each point from a real sensor in the
dynamic alignment process mainly because the uncertainty model for a point is
static and invariant and it is hard to describe the change of these physical
uncertainty models in the registration process. Additionally, the existing
Gaussian mixture alignment architecture cannot be efficiently implement these
dynamic changes.
This paper proposes a simple architecture combining error estimation from
sample covariances and dual dynamic global probability alignment using the
convolution of uncertainty-based Gaussian Mixture Models (GMM) from point
clouds. Firstly, we propose an efficient way to describe the change of each 3D
uncertainty model, which represents the structure of the point cloud much
better. Unlike the invariant GMM (representing a fixed point cloud) in
traditional Gaussian mixture alignment, we use two uncertainty-based GMMs that
change and interact with each other in each iteration. In order to have a wider
basin of convergence than other local algorithms, we design a more robust
energy function by convolving efficiently the two GMMs over the whole 3D space.
Tens of thousands of trials have been conducted on hundreds of models from
multiple datasets to demonstrate the proposed method's superior performance
compared with the current state-of-the-art methods. The new dataset and code is
available from https://github.com/Canpu999Comment: Accepted by 3DV 2018. 9 pages. arXiv admin note: text overlap with
arXiv:1707.0862
Best Viewpoint Tracking for Camera Mounted on Robotic Arm with Dynamic Obstacles
The problem of finding a next best viewpoint for 3D modeling or scene mapping
has been explored in computer vision over the last decade. This paper tackles a
similar problem, but with different characteristics. It proposes a method for
dynamic next best viewpoint recovery of a target point while avoiding possible
occlusions. Since the environment can change, the method has to iteratively
find the next best view with a global understanding of the free and occupied
parts.
We model the problem as a set of possible viewpoints which correspond to the
centers of the facets of a virtual tessellated hemisphere covering the scene.
Taking into account occlusions, distances between current and future
viewpoints, quality of the viewpoint and joint constraints (robot arm joint
distances or limits), we evaluate the next best viewpoint. The proposal has
been evaluated on 8 different scenarios with different occlusions and a short
3D video sequence to validate its dynamic performance.Comment: 10 pages, 6 figures, poster in 3DV conferenc
SDF-MAN: Semi-supervised Disparity Fusion with Multi-scale Adversarial Networks
Refining raw disparity maps from different algorithms to exploit their
complementary advantages is still challenging. Uncertainty estimation and
complex disparity relationships among pixels limit the accuracy and robustness
of existing methods and there is no standard method for fusion of different
kinds of depth data. In this paper, we introduce a new method to fuse disparity
maps from different sources, while incorporating supplementary information
(intensity, gradient, etc.) into a refiner network to better refine raw
disparity inputs. A discriminator network classifies disparities at different
receptive fields and scales. Assuming a Markov Random Field for the refined
disparity map produces better estimates of the true disparity distribution.
Both fully supervised and semi-supervised versions of the algorithm are
proposed. The approach includes a more robust loss function to inpaint invalid
disparity values and requires much less labeled data to train in the
semi-supervised learning mode. The algorithm can be generalized to fuse depths
from different kinds of depth sources. Experiments explored different fusion
opportunities: stereo-monocular fusion, stereo-ToF fusion and stereo-stereo
fusion. The experiments show the superiority of the proposed algorithm compared
with the most recent algorithms on public synthetic datasets (Scene Flow,
SYNTH3, our synthetic garden dataset) and real datasets (Kitti2015 dataset and
Trimbot2020 Garden dataset).Comment: This is our draft and accepted by the journal Remote Sensing. There
is a little difference between the title on Arxiv and that on Remote Sensing.
Two small corrections have been made in "Performance on Kitti2015 Dataset" in
this latest version (which is slightly different from the previous version in
Remote Sensing
Hybrid Multi-camera Visual Servoing to Moving Target
Visual servoing is a well-known task in robotics. However, there are still
challenges when multiple visual sources are combined to accurately guide the
robot or occlusions appear. In this paper we present a novel visual servoing
approach using hybrid multi-camera input data to lead a robot arm accurately to
dynamically moving target points in the presence of partial occlusions. The
approach uses four RGBD sensors as Eye-to-Hand (EtoH) visual input, and an
arm-mounted stereo camera as Eye-in-Hand (EinH). A Master supervisor task
selects between using the EtoH or the EinH, depending on the distance between
the robot and target. The Master also selects the subset of EtoH cameras that
best perceive the target. When the EinH sensor is used, if the target becomes
occluded or goes out of the sensor's view-frustum, the Master switches back to
the EtoH sensors to re-track the object. Using this adaptive visual input data,
the robot is then controlled using an iterative planner that uses position,
orientation and joint configuration to estimate the trajectory. Since the
target is dynamic, this trajectory is updated every time-step. Experiments show
good performance in four different situations: tracking a ball, targeting a
bulls-eye, guiding a straw to a mouth and delivering an item to a moving hand.
The experiments cover both simple situations such as a ball that is mostly
visible from all cameras, and more complex situations such as the mouth which
is partially occluded from some of the sensors.Comment: 6 pages, Published in IROS 201
Real-time Stereo Visual Servoing for Rose Pruning with Robotic Arm
The paper presents a working pipeline which integrates hardware and software in an automated robotic rose cutter. To the best of our knowledge, this is the first robot able to prune rose bushes in a natural environment. Unlike similar approaches like tree stem cutting, the proposed method does not require to scan the full plant, have multiple cameras around the bush, or assume that a stem does not move. It relies on a single stereo camera mounted on the end-effector of the robot and real-time visual servoing to navigate to the desired cutting location on the stem. The evaluation of the whole pipeline shows a good performance in a garden with unconstrained conditions, where finding and approaching a specific location on a stem is challenging due to occlusions caused by other stems and dynamic changes caused by the win